Hi @christophehurpeau, thank you for the contribution! We are going to discuss this change internally as there are some usability and necessity questions arose. Can you maybe share the exact use-case or small snippet to demonstrate usage scenario, this would be extremely helpful.

@tadjik1

Of course.

Most of our models include an schema with non-nullable fields. Here a simple example:

interface EndUser {
  _id: Types.ObjectID;
  fullname: string;
  email?: string;
}

With an index on email.
When we want to query all enduser with existing emails:

const endUsers = await EndUserQueries.find({ email: { $ne: null } })

Hi @christophehurpeau,
thanks for providing that snippet - it makes the context and the goal of this change much clearer.

While I understand the convenience this would provide from the application side, we have to consider the driver’s role as a transparent interface to the database. There is a fundamental distinction between a field being null and a field being missing.
Since your schema defines email as optional (essentially string | undefined), the idiomatic way to query for this in MongoDB is using $exists, for example { email: { $exists: true } } - to find all users with email.
If you do have users with their emails null in the collection, it might be best to reflect that in your interface:

interface EndUser {
  ...
  email: string | null;
}

Please let me know if this helps!

Hi @christophehurpeau, thanks for providing that snippet - it makes the context and the goal of this change much clearer.

While I understand the convenience this would provide from the application side, we have to consider the driver’s role as a transparent interface to the database. There is a fundamental distinction between a field being null and a field being missing. Since your schema defines email as optional (essentially string | undefined), the idiomatic way to query for this in MongoDB is using $exists, for example { email: { $exists: true } } - to find all users with email. If you do have users with their emails null in the collection, it might be best to reflect that in your interface:

interface EndUser {
  ...
  email: string | null;
}

Please let me know if this helps!

From what I understood, $exists did not use indexes. that's why we did not use it. It seems the bug was fixed, or for some other reason it works now, at least in the case I tested. I guess we can use $exists then. Thank you for your response.